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Abstract 


Cost, schedule, and quality may not drive a technology, but they shape the chances of that 
technology becoming actualized. In recent years, the DoD, one of the leading customers of 
unmanned systems, has continued to struggle with management of cost and schedule 
causing programs to deliver products that are “good enough,” delayed months to years, or 
even worse, decommissioned. Cost estimation techniques in use today are vast and based 
on techniques unrelated to emergent systems. One of the most prevalent requirements in the 
unmanned systems arena is autonomy. The acquisition community will need to adopt new 
methods for estimating the total cost of ownership of this new breed of systems. Singularly 
applying traditional software and hardware cost models do not provide this capability because 
the systems that were used to create and calibrate these models were not Unmanned 
Autonomous Systems (UMASS; Valerdi, Merrill, & Maloney, 2013). Autonomy, although not 
new, will redefine the entire way in which estimates are derived. The goal of this paper is to 
provide a method that attempts to account for how cost estimating for autonomy is different 
than current methodologies and to suggest ways it can be addressed through the integration 
and adaptation of existing cost models. 


Introduction 


Life Cycle Models 


When designing a product, the recommended practice is to consider design 
decisions and their impact throughout the entire life cycle. This is a holistic approach that 
allows the engineer to examine all phases, and ensure that the stakeholders’ (e.g., 
operators, testers, and maintainers) needs are met (Blanchard & Fabrycky, 2010). This is 
the same approach that should be taken when identifying product costs, thinking holistically 
throughout the life cycle. For purposes of discussing the realm of Unmanned Autonomous 
Systems (UMASs) we focus on two life cycle standards: DoD 5000 (Hagan, 2011; Mills, 
2014) and ISO/IEC 15288 Systems Engineering—System Life Cycle Processes (ISO/IEC, 
2002). 


Both product life cycle standards are organized into discrete phases. Each phase 
has a distinct role in the life cycle and helps separate major milestones throughout the life 
cycle of a product. These life cycle stages help answer the ‘when’ and are useful in 
identifying development, production, and operational costs. 
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DoD 5000 Acquisition Life Cycle 

Although there are many commercial customers being identified and pursued within 
the UMAS arena, the largest acquirer of autonomous systems is the DoD. The DoD 5000 is 
a useful framework to apply to a product, as it forces engineers to produce specific sub- 
products in each of the five phases (Hagan, 2011): 


1. Inthe first phase, Materiel Solution Analysis, the DoD requires an initial 
capabilities document and an analysis of alternatives study. 

2. During the second phase, Technology Development, the goals are to produce 
a demonstrable prototype that will allow the customer to make decisions in 
the risk, technology, and design. 

3. The third phase, Engineering and Manufacturing Development, forces the 
engineer to again demonstrate prototype articles, conduct integrated testing 
(Developmental, Operational, and Live Fire Test and Evaluation), Prepare for 
both the Critical Design Review and the proposal for product continuation. 

4. During the fourth phase, Production and Deployment, engineers are now 
preparing low-rate and full scale production. 

5. The final phase, Operations and Support, consists of activities such as 
maintaining capabilities, logistical support, upgrades, customer satisfaction, 
and prepare for proper disposal. 

The five phases and major milestones are shown in Figure 1. 
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Figure 1. DoD 5000 Acquisition Framework 
(Spainhower, 2003) 


ISO 15288 Life Cycle 

A definition of the system life cycle phases is needed to help define the boundaries 
between engineering activities. A useful standard is ISO/IEC 15288 Systems Engineering 
System Life Cycle Processes (ISO/IEC 15288). However, the phases established by 
ISO/IEC 15288 were slightly modified to reflect the influence ANSI/EIA 632 Processes for 
Engineering a System has on COSYSMO's System Life Cycle Phases, and are shown in 
Figure 2. 
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Figure 2, COSYSMO System Life Cycle Phases 


Life cycle models vary according to the nature, purpose, use, and prevailing 
circumstances of the product. Despite an infinite variety in system life cycle models, there is 
an essential set of characteristic life cycle phases that exists for use in the systems 
engineering domain. 


1. The Conceptualize stage focuses on identifying stakeholder needs, exploring 
different solution concepts, and proposing candidate solutions. 

2. The Development stage involves refining the system requirements, creating a 
solution description, and building a system. 

3. The Operational Test & Evaluation stage involves verifying/validating the 
system and performing the appropriate inspections before it is delivered to 
the user. 

4. The Operate, Maintain, or Enhance involves the actual operation and 
maintenance of the system required to sustain system capability. 

5. The Replace or Dismantle stage involves the retirement, storage, or disposal 
of the system. 

We revisit these life cycle models later in this section and decompose various types 
of costs into their respective phases to demonstrate Total Cost of Ownership. 


Cost Estimation Methods 


The exploration of new cost modeling methods involves the understanding of the 
cost metrics relevant to the UMAS as well as an understanding of their sensitivity to cost 
from a production and operational standpoint. In this light, this section provides an overview 
of different cost estimation approaches used in industry and government. Significant work 
has been done to understand the costs of aircraft manufacturing (Cook & Grasner, 2001; 
Markish, 2002; Martin & Evans, 2000) but these studies only deal with manned commercial 
and military aircraft. Nevertheless, they provide useful insight on how one could approach 
the estimation of the UMAS life cycle cost. 


Case Study and Analogy 
Recognizing that companies do not constantly reinvent the wheel every time a new 
project comes along, there is an approach that capitalizes on the institutional memory of an 
organization to develop cost estimates. Case studies represent an inductive process 
whereby estimators and planners try to learn useful general lessons by extrapolation from 
specific examples. They examine in detail elaborate studies describing the environmental 
conditions and constraints that were present during the development of previous projects, 
the technical and managerial decisions that were made, and the final successes or failures 
that resulted. They then determine the underlying links between cause and effect that can 
be applied in other contexts. Ideally, they look for cases describing projects similar to the 
project for which they will be attempting to develop estimates and apply the rule of analogy 
that assumes previous performance is an indicator of future performance. The sources of 
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case studies may be either internal or external to the estimator's own organization. Home- 
grown cases are likely to be more relevant for the purposes of estimation because they 
reflect the specific engineering and business practices likely to be applied to an 
organization's projects in the future. Well-documented case studies from other organizations 
doing similar kinds of work can also prove very useful so long as their differences are 
identified. 


Bottom-Up & Activity-Based 
Bottom-up estimating begins with the lowest level cost component and rolls it up to 
the highest level for its estimate. The main advantage is that the lower level estimates are 
typically provided by the people who will be responsible for doing the work. This work is, 
typically represented in the form of subsystem components, which makes this estimate 
easily justifiable because of their close relationship to the activities required by each of the 
system components. This approach also allows for different levels of detail for each 
component. For example, the costs of an airplane can be broken down into seven main 
components: center-body, wing, landing gear, propulsion, systems, payloads, and 
assembly. Each of these components, such as the wing, can be decomposed into 
subcomponents such as winglet, outer wing, and inner wing. This decomposition is 
illustrated in more detail in Figure 3. This can translate to a fairly accurate estimate at the 
lower level components. The disadvantages are that this process is labor intensive and is 
typically not uniform across products. In addition, every level introduces another layer of 
conservative management reserve which can result in an overestimate at the end. 
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Figure 3, Product Breakdown Structure of a Typical UMAS 


Parametric Modeling 
This method is the most sophisticated and most time consuming to develop but often 
provides the most accurate result. Parametric models generate cost estimates based on 
mathematical relationships between independent variables (i.e., requirements) and 
dependent variables (i.e., effort or cost). The inputs characterize the nature of the work to be 
done, plus the environmental conditions under which the work will be performed and 
delivered. The definition of the mathematical relationships between the independent and 
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dependent variables is the heart of parametric modeling. These relationships are commonly 
referred to as cost estimating relationships (CERs) and are usually based upon statistical 
analyses of large amounts of data. Regression models are used to validate the CERs and 
operationalize them in linear or nonlinear equations. The main advantage of using 
parametric models is that, once validated, they are fast and easy to use. They do not require 
a lot of information and can provide fairly accurate estimates. Parametric models can also 
be tailored to a specific organization's characteristics such as productivity rates, salary 
structures, and work breakdown structures. The major disadvantage of parametric models is 
that they are difficult and time consuming to develop and require a lot of clean, complete, 
and recent data to be properly validated. Despite the wide range of estimation approaches 
available for commercial and military aircraft, no parametric models have been created 
specifically for a UMAS. This could be attributed to the fact that UMASs have not been 
around for very long and, as a result, there are insufficient data available to validate such 
models. Before proposing a framework for such a model, unique issues pertaining to the 
UMAS life cycle are discussed. 


UMAS Product Breakdown Structure 

It is widely recognized that creating a work breakdown structure (WBS) or product 
breakdown structure (PBS) is the most complete way to describe a project (Larson, 1952). 
The level of detail required to properly utilize, or manage with, the PBS such as the one 
shown in Figure 3 is a crucial component to assigning costs to a product's subcomponents. 
In this section, we discuss some of the commonalities and shared considerations of 
designing a WBS/PBS within an unmanned system at the system level. Budgeted amounts 
for various unmanned and autonomous systems are shown in Tables 14 at the 2nd or 3rd 
level of a WBS/PBS. 


Table 1. Air System (UAS) 








(DoD, 2014a) 
Unmanned Aerial Vehicle — Unit Cost Number | Total Cost | Program 
Global Hawk (SM) of Units | _($M)__| allocation* 
‘Aerial Vehicle 69.84 45 3143.16 | 66.60% 
Ground Control Station 21.82 10 218.21 | 4.62% 
Support Element nla na 1,357.84 | 28.77% 
Projected Total Cost nla wa | 4,719.21 | 100.00% 

















*Since the program allocation was only available for the Global Hawk, we applied the same ratios to other 
unmanned programs. 


Table 2. Ground System (UGS) 
(DoD, 2014b) 








Unmanned Ground System Unit Cost Number Total Cost 
COTS/GOTS (Ms) of Units (SM) 
Ground Vehicle 3.39 4 13.56 
Ground Control Station 0.23 4 0.94 
Support Element na nla 5.86 
Projected Total Costs na nla 20.36 
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Table 3. Ground System (UGS) 
(DoD, 2014b) 








‘Small Unmanned Ground Unit Cost] Number of Units Total Cost 
({SUGV) (MS) (SM) 
Ground Vehicle 0.180 aT 35.90 
Ground Control Station 0.012 a 03.88 
Support Element nla nla 24.15 
Projected Total Costs na nla 83.93 














Table 4. Marine System (UGS) 
(DoD, 2014e) 








Modular Unmanned Unit Cost | Number of Units Unit Cost 
Scouting Craft Littoral (Ms) (sm) 
(MUSCL) 

Maritime Vehicle 0700 Ex} 503 
Surface Control Station 0.048 13 0.62 
Support Element na nla 3.90 
Projected Total Costs nla nla 13.56 














That Ground Control Stations are the user controls (i.e., the video game-like interface to maneuver 
vehicle) 

“Italicized numbers = extrapolation based off of RQ-4 Global hawk program ratio 

“unaltered numbers are from the Exhibit P-40 Presidential Budget FY2015 or equivalent cost data 


One observation from the UMAS examples provided in Tables 1-4 is the range of 
unit costs. On the high end, the Flyaway Unit Cost of the Global Hawk Unmanned Aircraft 
System is $92.87 million (DoD, 2014a, p. 177). On the low end, the Modular Unmanned 
Scouting Craft Littoral is $700,000 (DoD, 2014c). Another observation from these examples 
is the wide range of units purchased; as few as four COTS/GOTS packages to convert 
manned systems to unmanned and as many as 311 Small Unmanned Ground Systems 
(DoD, 2014b). 


Special Considerations 

The unique physical and operational characteristics of UMASs require special 
consideration when exploring cost modeling approaches. In Figure 4, the DoD has laid out 
its desires for the UMAS over the next 30 years. The DoD has organized its requirements by 
air, ground, and maritime operational environments, as well as projected the types of 
exploration initiatives that should allow for success of these autonomous systems. Figure 4 
is not meant to be totally exhaustive, but to guide the general direction of the military's 
UMAS vision. 
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Figure 4. Operating Environment Technology Development Timeline (2013-2030) 
(DoD, 2013) 


Mission Requirements (DoD, 2013) 

The mission requirements are specified tasks with which the UMAS must comply in 
order to perform. These requirements are shaped by the operational environment (OE), or 
venue by which the UMAS will perform its intended functions or capabilities that can be 
physical and situational. The physical environment can consist of air, ground (surface and 
sub-surface), and marine (surface and submersible.) 


System Capabilities 
In essence, what will the UMAS do for the customer? These functions must also 
include current capabilities such as attack, logistical, and reconnaissance. This area also 
includes any of the “ilities" that a UMAS might need to adhere to that are not specified in its 
mission requirements. These may include manufacturability, reliability, interoperability, 
survivability, and maintainability. 
Payloads 
A final consideration for the UMAS is its payload. This could also be categorized as 
special equipment. For example, a logistical UMAS (or cargo transportation system like the 
SMSS™) needs to have a tow system or recovery package in addition to the ground vehicle; 
or if it is an attack/reconnaissance system—it needs to support munitions, missiles, or gun 
platforms. 


Although many more areas can be identified for consideration when engineering a 
system for autonomy, this section was meant to highlight the WBS/PBS in more detail rather 
than the technical capabilities of the UMAS itself. The cost to build and produce a system is 
a bottom line decision for the producer (and the engineer), but the DoD needs and expects 
that a WBS represent all phases of the life cycle. By accurately representing the system in a 
more complete WBSIPBS, the cost estimates will have more fidelity and a higher 
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confidence, because estimators will be able to link the lowest level of that structure to a 
group of cost drivers within a cost model. 


Cost Drivers and Parametric Cost Models 

Cost drivers are characteristics of projects that best capture the effort, typically 
measured in Person Months, required to complete them (Boehm, 2000). As mentioned in 
the Parametric Modeling section, developing these characteristics, or drivers, is data and 
labor intensive. The developer of the model must establish a strong mathematical 
relationship, usually a form of regression, between an identified characteristic and its impact 
on the project. The number of cost drivers for each type of estimate will vary according to 
the type of component (hardware, software, etc.). 


Each cost driver has a scale, usually of five levels, which allows the user of the 
model to best represent characteristics of the product. For example, a cost driver can be 
described using Very Low, Low, Nominal, High, or Very High—each one of these choices 
has a value that will either increase or decrease cost (Valerdi, 2008). Each level is clearly 
defined so the user can estimate the complexity of a system as realistically as possible. The 
key for success with utilizing parametric modeling and its drivers is to fully understand and 
be realistic with assignment of scale values. 


Cost Drivers for Estimating Development Costs 
Our proposed method for system level estimation is to combine five different 
parametric models that best represent the amount of effort required to successfully build, 
test, produce, and operate an Unmanned Autonomous System (UMAS). These include (1) 
Hardware, (2) Software, (3) Systems Engineering and Program Management, (4) 
Performance-Based Characteristics, and (5) Weight-Based Characteristics. 


Each of the five models is subsequently described and should be considered when 
developing a complete life cycle estimate; however, it is not mandatory to utilize all five since 
each UMAS will have unique cost and performance considerations. 


Hardware 
SEERH is a hybrid model that utilizes analogous estimates, as well as hamessing 
parametric mathematical cost estimation relationships specific to hardware products. SEER- 
H aids in the estimation of hardware development, production, and operations costs (SEER- 
H® Documentation Team: MC, WL, JT, KM, 2014). Unlike the other estimation tools 
available, SEER-H has an exhaustive suite and could be used to estimate many technical 
areas. The number of cost drivers in SEER-H is extensive; therefore we focus on only three 
within the Mechanical/Structural Work Elements category: 
* Material Composition—the material that will dominate the system and its 
difficulty to acquire 
* Certification Level—the amount of Test & Evaluation with demonstration 
required for the materials utilized 
* Production Tools and Practices—how ready the materials are for production 


Material Composition 

This SEER-H driver is categorized by the predominant material used to build the 
system, sub-system, or the system's components, as shown in Table 5. The estimator 
should also consider some of the materials that may not dominate, but are identified as 
critical. The total cost may be a combination of critical and dominant materials. 
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Table 5. Material Composition Rating Scale 
(SEER-H® Documentation Team, 2014) 


Key Property 
Metal alloy, easily manufactured 

Example: Aluminum, magnesium, copper, aluminum-lithium. 

Hard, rigid metal alloy, resistant to rust. 

Example: Steel, Stainless Steel. 

Commercially Available | Commodity available exotic materials. 

Exotic Example: Titanium, precious metals, boron, higher end composites. 
Other Exotic Requires very complex metallurgical processes, available only through 
special orders. 

Example: Metal matrix composites, particulate strengthened composite 
materials, research materials. 

Composite ‘Commodity available, continuous filament or particulate strengthened 
composite materials, 

Example: Graphite or boron epoxy, fiber glass. 
































Polymer Nonmetallic compound, easily molded, may be hardened or pliable. 
Example: Plastics, thermoplastics, elastomers. 
Ceramic Very Strong, brittle 





Example: Ceramic, clay, glass, tile, porcelain 





Certification Level 
Certification level represents the requirements imposed on the manufacturer by the 
customer, as shown in Table 6. This parameter quantifies the additional cost associated with 
the customer's certification requirements; therefore, any extra certification, inspections, or 
intangible property security controls, etc., will increase cost. 
Table 6. Certification Level Rating Scale 
(SEER-H® Documentation Team, 2014) 





Rating Description 
VERY HIGH | Very high level of qualification testing including fatigue, fracture mechanics, burst, 
temperature extremes and vibration testing, Example: Manned Space Product 

HIGH High level of qualification testing including fatigue, fracture mechanics, burst, 
temperature extremes and vibration testing. Example: Space Product. 

NOMINAL + | Quaiffcation testing for mission requirements including static and dynamic load testing, 
\wind tunnel testing and all other tests required for military aircraft. Example: Military 
Aiborne! Aircraft Product 

NOMINAL | Qualification testing in accordance with FAA requirements, as specified for commercial 
co general aviation aircraft. Example: Airborne? Aircraft Product. 

NOMINAL | Qualification testing in accordance with U.S. Army Mobility requirements, or US. Navy 
specifications. Testing includes meeting shock, vibration, temperature and humidity 
requirements. Example: Military Ground-Mobile or Sea Product, 

tow Nominal qualification testing for mission requirements covering equipment located in 
controlled envitonments (temperature, humidity). Example: Military Ground System 
VERY LOW_| Minimal testing required (functional check-out). Example: Commercial Grade Product 





























Production Tools and Practices 

This parameter describes the extent to which efficient fabrication methodologies and 
processes are used, and the automation of labor-intensive operations. The rating should 
reflect the state of production tools that are in place and already being used by the time 
hardware production begins (see Table 7). 
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Table 7. Production Tools and Practices Rating Scale 
(SEER-H® Documentation Team, 2014) 


Rating Description 

VERY HIGH | Production tooling associated normally with large-scale production (20,000 units or 
above). Highly sophisticated tools, die casts, molds. High degree of mechanization, 
robotics manufacture, assembly and testing. High degree of integration between 
computer-aided manufacturing and design. Example: Die casting, multi-cavity molds, 
progressive dies and other sophisticated tools. 

HIGH Production tooling normally applied to medium scale, averaging 20,000 unit production, 
Tools are custom designed with simple dies. Some degree of mechanization, 
numerically controlled machine tools, some integration with computer aided design. 
Example: Simple die casting, complex investment casting, custom die sets for sheet 
metal fabrication 

NOMINAL | Production tooling facilitates production of 1,000-2,000 units. Complex tools, simple 
dies and castings. Little mechanization, few numerically controlled machining 
operations. Some automated links with CAD. Example: Complex sand castings, 
investment castings of some complexity, and simple custom die sets are included in the 
tooling category. 

tow Tooling designed for the production of up to 7,000 units. Standard tools, casts, dies, 
and fixtures are supplemented with some custom tools and jigs. Occasional or 
experimental use of automated links with CAD. Example: Sand castings, investment 
castings and simple custom die sets, Many Aerospace/ DoD programs are in this 
category 

VERY LOW | Minimum tooling required to produce up to about 50-100 units, Many operations of 
manufacture, assembly and test are by skilled labor. The use of standard tools and 
fixtures is predominant. No automated links. Example: Simple sand castings are in this 


category. 










































Software 

The recommended parametric estimation tool for UMAS software aspects is the 
Constructive Cost Model (COCOMO 1). This model has 30 years of refinement, and is an 
industry and academic standard for parametric modeling (Boehm, 2000). The number of 
cost drivers in COCOMO II vary from 7 to 17 depending on the life cycle phase of the project 
in which the estimate is being performed (Boehm, 2000). Since less information is known at 
the beginning of the project, the COCOMO II model provides fewer parameters to rate. As 
more information is known about the software project, the number of parameters increases. 
This section is not meant to replace the COCOMO II User's Manual,’ but rather provide 
relevant details about the relevant cost drivers. Three drivers are relevant for UMAS 
software estimation: 


*  Size—measured by number of lines of lode 
* Team Cohesion—weighted average of four characteristics 
* Programmer Capability—how efficient programmers are as a whole 
Size 


Size is in units of thousands of source lines of code (KSLOC) is derived from 
estimating the size of software modules that will constitute the application program. It can 
also be estimated using unadjusted function points (UFP), converted to SLOC, then divided 
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by one thousand. Equation 1 is the basic COCOMO | algorithm which includes Size as the 
central component to calculating effort in Person Months (PM). 


pM = ax<size) x] [EM (1) 
ist 


Team Cohesion 

This parameter accounts for the human component in software design. These 
elements are not limited to but contain differences in multiple stake-holder objectives, 
cultural backgrounds, team resiliency, and team familiarity (see Table 8). The focus is how 
the design team interacts externally within the project. 


Table 8, Team Cohesion Rating Scale 





Characteristic Very Low Low Nominal High Very High Extra High 
Consistency of Stakeholder | Litile ‘Some Basic Considerable Strong Full 
objectives and cultures 








‘Ability, wilingness of Title Some Basic Gonsidarable Strong Full 
‘stakeholders to accommodate 
‘other stakeholders objectives 














Experience of stakeholders in] None Lie tie Basic Conderabie —Eenawve 
‘operating as a team 
Stakeholder teambuilding te | None Lila —Litje Basie Considerable —Exienawve 
achieve shared vision and 
commitments 

Programmer Capability 


This parameter also deals with a human aspect of software engineering; however, it 
differs from team cohesion in the direction of the focus. In this parameter the assessment is 
on the internal workings of the team’s capability as it relates to the team's efficiency, 
thoroughness, internal communication, and cooperation (see Table 9). 


Table 9. Programmer Capability Rating Scale 











PCAP Descriptors 15" 358 3 75® percentile 90" 

percentile percentile percentile percentile 
Rating Levels ‘VeryLow Low Nominal High Very High Extra High 
Effort Multipliers 1.34 115 1.00 0.88 0.76 ala 





Systems Engineering and Project Management 

To estimate the Systems Engineering and Project Management required effort for a 
UMAS, we use the Constructive Systems Engineering Cost Model (COSYSMO). This 
parametric model's output accounts for integrating system components and will quantify 
intangible efforts such as requirements, architecting, design, verification, and validation 
(Valerdi, 2008). This model also depends on 18 size and cost drivers.? By introducing some 
of the most important drivers we capture the most important cost considerations of a UMAS. 
The three most relevant systems engineering cost drivers are as follows: 


2 http://cosysmo.mit.edu 
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* Number of System Requirements—number of specified functions a system 
must perform to meet the user's needs 
* Technology Risk—how mature or demonstrated the technologies are 
* Process Capability—how well/consistent the team/organization performs in 
terms of the Capability Maturity Model Integration (CMMI) 
Number of Requirements 
The Number of Requirements parameter asks the estimator to count the number of 
requirements for the UMAS at a specific level of design (see Table 10). These requirements 
may deal with number of system interfaces, system specific algorithms, and operational 
scenarios. Requirements are not limited to but may be functional, performance, feature, or 
service-oriented in nature depending on the methodology used for specification. Of note, 
requirement statements usually contain the words “shall,” “will,” “should,” or “may.” 


Table 10. Number of Requirements Rating Scale 























Easy Nominal Dificul 
Simple to implement Familiar Complex to implement 
Traceable to source Can be traced to source Hard to trace to source 

with some effort 
ile requirements ‘Some overlap High degree of requirement 
overlap overlap 
Technology Risk 


The Technology Risk parameter asks you to evaluate a UMAS's sub-system's 
maturity, readiness, and obsolescence of the technologies being implemented (see Table 
11). Immature or obsolescent technologies will require more systems engineering effort. 


Table 11. Technology Risk Rating Scale 

















Very Low Tow Nominal High Very High 
Tack of Technology | Proven through | Proven on pilot | Readyfor __| Stillin the 
Maturity proven and | actualuse and | projects and pilotuse | laboratory 
widely used | ready for ready to roll-out 
throughout | widespread for production 
industry adoption jobs 
Tack of Mission Concept Concept has Proofof | Concept 
Readiness | proven qualified been concept —_| Defined 
(TRL 9) (TRL 8) demonstrated | validated | (TRL3 & 4) 
(TRL 7) (TRL5 86) 

‘Obsolescence Technology i | Technology | Technology is 
the state-ofthe- | is stale outdated and 
practice New and | should be 
Emerging avoided in new 
technology could systems 
compete in Spare parts 
future pilotuse | supply is 

scarce 




















Process Capability 

Like some of the COCOMO II parameters, this COSYSMO example focuses on the 
consistency and effectiveness of a project team performing the systems engineering 
processes. The assessment of this driver may be based on ratings from a published process 
model (e.g., CMMI [2002], EIA-731 [ANSI/EIA, 2002], SE-CMM [Boehm, 2000; Clark, 1997], 
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ISO/IEC 15504 [2003, 2012). It can alternatively be based on project team behavioral 
characteristics if no previous external assessments have occurred. 


Table 12. Process Capability Rating Scale 

















Very Low | tow Nominal High Very High | Extra High 
— | Level 0(iF | Levelt Level 2 Level 3 Level 4 Level 5 
% | continuous 
£ | model) 
Es 
24 
2 
Ad Hoc Performed | Managed SE | Defined SE | Quantitatively | Optimizing 
approach to | SE process, | process, process, Managed SE | SE process, 
process activities activities activities process, continuous 
8 performance | driven only by | driven by driven by | activities improvement, 
3 immediate | customer and | benefitto | driven by SE_| activities 
3 Contractual or | stakeholder | project, 3 | benef, 8 | driven by 
g customer |needsina. | focusis | focuson all_| system 
: requirements, | suitable through phases of the | engineering 
& SE focus | | manner, SE | operation, _| life cycle and 
3 limited focus is process organizational 
3 requirements | approach benefit, SE 
2 through driven by focus is 
2 design, organizational product life 
= project-centric | processes cycle & 
5 approach— | tailored for strategic 
e riot driven by | the project applications 
z organizational 
2 processes 
Wanagement | SEMP Project uses a | Highly The SEMPis | Organization 
judgmentis | usedinan | SEMP with | customized | thorough and_| develop best 
used ad-hoc some SEMP exists | consistently | practices for 
manner only | customization | andisused | used: SEMP; all 
§ ‘on portions of throughout ‘organizational | aspects of the 
5 the project the rewards are | project are 
$ that require it organization | in place for | included in 
2 those that | the SEMP: 
é improve it | organizational 
a reward exists 
z for those that 
a improve it 























Performance-Based Cost Estimating Relationship 

One important consideration of every product is its ability to perform the specified 
requirements well. The model that best captures the performance characteristics of a 
product was created by the Army for Unmanned Aerial Vehicle Systems, but can be 
modified to fit other autonomous systems (Cherwonik & Wehrley, 2003). The methodologies 
for estimating performance are not restricted to this list, but should fit in similar categories for 
air, land, sea, or space (see Table 13). 
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Table 13. Performance-Based Characteristics Rating Scale 
(Cherwonik & Wehrley, 2003) 














Performance Based Categories | Descriptions 

Vehicle or Body of UMAS. Define and measure how well the vehicle or body of the 
UMAS performs its intended requirements. 

‘Sensors Define and measure how well the UMAS can interact and 
react with its intended (or unintended) environment 

Control System Define and measure how efficient the command and control 
system interacts with UMAS. 








The cost drivers that are recommended for performance measurement are based on 
an aerial platform, but are modified in this section to provide ideas on what areas to consider 
(see Table 14). 

Table 14, Performance Cost Drivers 
(Cherwonik & Wehrley, 2003) 




















Performance Drivers Description’ Use of driver 

‘Operational Environment Constraints | Define and measure the physical boundaries guiding the 
UMAS. 

Endurance Define and measure the amount of time or distance the UMAS. 
can perform its intended task prior to needing human 
interaction 

‘Sensor Resolution Define and measure the sensitivity, accuracy, resiliency, and 
efficiency of the UMAS sensors 

Base of Operations Define and measure how constrained the UMAS by its 
logistical requirements and the resources required for effective 
‘operations 








The Army's performance-based Cost Estimating Relationship is shown in Equation 2: 


JAV TAR4 (FYO3SK) =118.75 * (Endurance"Payload-Wt.)952” * @D010EE veae1200 * g 
92113 vO) 
Where: 

JAV T1R1 = Theoretical first unit cost of UAV air vehicle hardware normalized for 
learning (95% slope) and rate (95% slope), via unit theory. In FY03 SK. 
Endurance = UAV air vehicle endurance in fight hours 
Payload-Wt. = Weight of total payload in pounds. Total payload includes all 
equipment other than the equipment that is necessary to fly and excludes fuel and 
weapons 
FF-Year = 
Prod 1/0 


Year of first flight 
1 if air vehicle is a production unit 
if air vehicle is a development or demonstration unit (2) 









Weight-Based Cost Estimating Relationship 

A final consideration for estimating the cost of the UMAS is its weight. Weight may 
already exist as an important cost driver in other estimation models such as hardware and 
performance; however, we feel that this particular estimation relationship is strong enough to 
also be a stand-alone component. When operational implementation is considered for a 
given autonomous system, weight plays a critical role in the success or failure. Some 
drivers, modified from the source to apply to the UMAS, are shown in Table 15. 
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Table 15. Weight-Based Cost Drivers 
(Cherwonik & Wehrley, 2003) 


Weight-Based Drivers Description/Use of driver 
Weight of total system Define and Measure the weight of total system as it relates to 
its intended objectives (this does not include ordnance or 
other attachable options). 

















Payload Weight Define and Measure the amount and type of ordnance or any 

additional attachable option that is deemed mission critical 
Sling-load or Recovery Operation Define and Measure the amount of weight the UMAS can 
Capacity” support as a sling load or in a tow capacity, in addition to its 
“if applicable nominal capacity 





The Army's weight-based Cost Estimating Relationship is shown in Equation 3: 


UAV TAR1 (FY03SK) = 12.55 * (MGTOW) 974 * 237163 10) 
Where 

UAV T1R1 = Theoretical first unit cost of UAV air vehicle hardware normalized for 
learning (95% slope) and rate (95% slope). In FY03 SK 





MGTOW = UAV air vehicle maximum take-off weight in pounds. 
Prod 1/0 = 1 if air vehicle is a production system 
= 0 if air vehicle is a development or demonstration model. (3) 


Proposed Cost Drivers for DoD 5000.02 Phase Operations & Support 


Logistics—Transition From Contractor Life Support (CLS) for Life to Organic 
Capabilities 
Managing logistic support is complex and not easy to summarize into a single 
parameter. However, all systems require maintenance which can be described within the 
range provided in Table 16. The goal of this parameter is to allow life cycle planners to nest 
their system engineering plan into DoD requirements and minimize contractor life support. 


Table 16. Logistics Cost Driver 























Uniformed >2years 25 year <b year CLS Only 
Servicemen Only | transition transition transition 
‘System was Very few Few contractors | Contractors are | System is so 
designed ina —_| contractors (1-_ | (6-10) needed _| needed at every | technologically 
manner that 5)needed at | at Colonel (0-6) | level of advanced that 
current life Colonel (0-6) | level command | command operational use 
support is level command | unitsto ensure | Captain (0-3) _ | will require a 
sufficient for units to ensure | proper life through Colonel | permanent 
operational use. | proper life support (0-6). Minimum | contractor 
support Jat each level__| presence. 
Training 


The development costs for a UMAS can be significant, but one area of consideration 
is how quickly and efficiently users can be trained to employ the system. With the increasing 
levels of autonomy, this warrants its own cost driver. 
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Table 17. Training Cost Driver Considerations 








Minimalimpact ] Medium Impact | High Impact Extreme Impact] Unknown 
Impact 

Training fils Training program | Training program | Training program | Training 

current TRADOC? | is similar to a isnotsimilarto | is not similar to _| systems are 

through-put. And | current DoD any current DoD | any current DoD _| stil being 

requires minimal | method; however, | method. Needs to | method. Needs to | developed and 

certification needs to be a beastand-alone | bea stand-alone | will require 

(example system | stand-alone block | course.Needs | course. Needs _| extensive 

isa modified of instruction or | facilities and facilities and integration 

version of a course. Can use infrastructure not | infrastructure not 

previously existing facilities _| currently currently 

integrated and infrastructure | provided. available. 

systems — currently 

autonomous provided. 

raven) 

















The planning for and implementation of such training considerations in Table 17 will 
be challenging. The DoD acknowledges these challenges and offers a perspective of 
expectation management displayed in Figure 5. The training objectives attempt to lay out 
how the UMAS and other emergent systems will be inculcated into the existing training 
system. As engineers build their systems understanding, these strategies will help with 
system implementation in areas that are not implicitly the system being procured. 







20182019 





2020 2021 





n 2013, 


Technology 
Projects 


Capabilty 
Needs 


20142015 2036 2017, 




















Figure 5. UMAS Training Objectives (2013-2030) 
(DoD, 2013) 


Operations—Manned Unmanned Systems Teaming (MUM-T) 

The goal of the DoD’s investment in the UMAS is to enhance the warfighters’ 
capability while reducing risk to human life, maintaining tactical advantage, and performing 
tasks that can be dull, dirty, or dangerous (DoD, 2013).However, all of the systems will 
require some level of manned-with-unmanned cooperation. The more these two worlds 
efficiently work together, the better the operational outcome. 


2U.S. Army Training and Doctrine Command 
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Table 18. Manned Unmanned Systems Teaming Cost Driver 




















Very Low Tow Teaming Nominal Teaming | High Teaming | Very High 
Teaming Teaming 
Meets nojoint | Meets minimum | Meets branch Meets all branch | Meets ail Joint 
interoperability | branch specific | specific specific interoperability 
requirements, and | interoperal interoperability | interoperability ements, and 
generates data | requirements, but | requirements, and | requirements, as shares a common 
that needs tobe | is not compatible | shares information | well as one or —_| operating picture 
transferredto.a | with all systems | with manned more joint other manned and 
common employed by its | systems, branch | requirements unmanned 
operating picture | home branch specific. Also shares systems can 
information with | utilize 
manned systems, 














Considerations for Estimating Unmanned Ground Vehicle 


For a large scale project that requires the integration of multiple engineering 
disciplines, specifically in the field of the UMAS, no single estimation tool can completely 
capture total life cycle costs. By applying the proper estimation models, or a combination of 
these models, the estimator can ensure complete coverage of each program element and 
their relative cost impact across the UMAS project life cycle. 


The example used to illustrate the cost estimating process is the Lockheed Martin 
Unmanned Autonomous Ground System, Squad Mission Support System (SMSS™). By 
utilizing the product work breakdown structure (P-WBS) cost experts can then apply an 
estimation tool at the appropriate level. The sum of each sub-estimate is then integrated into 
the overall project level estimation. Considerations for which level within the P-WBS requires 
estimates is unique to each UMAS project. Contractual requirements will be the determining 
factor on how detailed the estimate needs to be. 


In response to the critical need for lightening, the soldier and marine infantryman’s 
load in combat as well as providing the utility and availability of equipment that could not 
otherwise be transported by dismounted troops, the Squad Mission Support System is being 
developed by Lockheed Martin. The SMSS™ can address the requirements of Light 
Infantry, Marine, and Special Operating Forces to maneuver in complex terrain and harsh 
environments, carrying all types of gear, materiel, and Mission Equipment Packages (MEP). 


The SMSS™ is a squad-sized UGV platform shown in Figure 6, about the size of a 
compact car, capable of carrying up to 1,500 pounds of payload. Designed to serve as a 
utility and cargo transport for dismounted small unit operations, it possesses excellent 
mobility in most terrains. The SMSS™/ Transport lightens the load of a 9-13 man team by 
carrying their extended mission equipment, food, weapons, and ammunition on unimproved 
roads, in urban environments, and on cross-country terrain. Control modes include tethered, 
radio control, teleoperation (NLOS and BLOS), supervised autonomy, and voice command. 
TRL level is 7-9. 
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Figure 6. Squad Mission Support System (SMSS™) 
(Lockheed Martin, n.d.) 


‘As shown in Table 19, the five proposed cost models adequately capture all of the P- 
WBS elements of the SMSS™. In some cases, the cost of individual elements can be 
captured by more than one cost model. To ensure that costs are not double counted, the 
estimator should decide which of the cost models will be used for each WBS element. This 


decision could be based on the amount of fidelity provided by each cost model or the ability 
of the cost model to capture the WBS element's characteristics that influence cost. 
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Table 19. Types of Estimates Needed per Product Breakdown Structure Element 










































































"Type of Model Recommended 
Rete | WES Bement Tardware | Sobware ‘Systems | Weight] Performance 
(Wits, 2014) (GEER) | (COCOMO'N) | Engineering | based | based CER 
(COSYEMO) | CER 
7 
Tit Venice Inteoration, * 
Assembly, Test, nd 
Checkout 
Tz FallFremelBodyiCab * Fy 
T7121 | Main Chassis Structure x £3 
TET | Frame and Pal * = 
Tiat2 | Hose Fy 
TASTS | Beck Panels * 
Tete | Sea Plate ¥ 
T7122 | Electrons Bax Srociure Fs 
T7123 | Front Brush Guard Fy 
Tea | Rear Srush Guard = 
THRE | Front SensoriCompanent * ¥ 
Mount 
T7128 | Rear Sensor Component * * 
Mount 
Tia? | Equpment Rack ¥ 
TARE | Pack RackalTal Gate * 
TS System Surivabity x ¥ 
Tie Turret Assembly % * = 
ro Taspeon Seeing = 
Te Vahice Blecronice % * 
Tar Foner Packagelorive Tran | x ¥ 
Te “Aasaery Automotive x x % ¥ 
Tae Fire Contra ¥ 
TAO _| Armament ¥ ¥ 
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THT] Automate Ammunition * x 
Hancing 

Tie | Navigation and Remote ¥ 
Piling 

Tatat | Navgaton Unt Fy ¥ 

Thea | Robotics Subsystem x x 

Thies | Autonomy Subsystem ¥ * 

Tia | Special Equipment Fs 

Tae | Communications % * Fs 

Tie | Venice Software Release FS 

Ti18 | Other Vehicle Subsystems = 

Tz Remote Contr System ¥ 

Tat Ramate Contrel System Fa ¥ 
Integration, Aacembly, Test, 
‘and Checkout 

Tae ‘Ground Cortal Canter ¥ 
Subsystem 

Exy ‘Operator Central Unt (OCU) ¥ 
‘Subsystem 

Tat Remote Contrel System x 
Sofware Release 























Once the appropriate cost models are determined for each WBS element, the cost 
can be calculated as the sum of the outputs of the five cost models, as shown in Equation 4. 


Cost (convert all individual outputs to $K) 


= (Hardware) + (COCOMO II) + (COSYSMO) (4) 
+(Weight Based CER) + (Perfomance Based CER) 


The expected unit cost would be in the range of $1 million to $100 million, depending 
on the capabilities and complexities of the UMAS. This is based on the historical results from 
the unit cost of the Global Hawk Unmanned Aircraft System ($92.87 million) and Modular 
Unmanned Scouting Craft Littoral ($700,000). If the estimated cost falls outside of this 
range, careful analysis should be done to ensure that the capabilities of the UMAS being 
estimated are truly beyond the scope of the historical data. 


Another basis of comparison could be the two cost estimating relationships 
described in this section which consider flight hours and maximum takeoff weight. While 
these cost drivers would only be relevant for Unmanned Aerial Vehicles, they can serve as 
sanity checks when performance and weight are important considerations. 


For the purposes of this section of the report, we are unable to provide a comparison 
of actual costs versus estimated costs to validate our proposed cost modeling approach. 
One reason is the proprietary nature of the data. Another is the lack of fidelity that is 
available to compare UMAS costs using the same cost elements, namely vehicle, ground 
control station, and support elements. 
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Additional Considerations for UMAS Cost Estimation 


Test and Evaluation 

Many systems engineering and project management experts advise concurrent 
planning of test and evaluation (T&E) during the earliest phases of a project (Blanchard & 
Fabrycky, 2003). In similar fashion, estimating the cost of these activities should also begin 
earlier rather than later. As budgets are allocated and costs are estimated, some key 
considerations on how the UMAS may be tested might be analytical testing, prototyping, 
production sampling, demonstration, and modification (Blanchard & Fabrycky, 2003). The 
current practice in many organizations is to focus most of the cost of product development 
and when the project reaches the T&E phases use the remaining funding. This often leads 
to reduced testing and schedule slippages. 
2 Demonstration 

Demonstration is one of the unique aspects of T&E because there are many 
categories or sub-sets of demonstrating a product's capability. The two that are most 
important are demonstrating systems integration and demonstrating full operational 
capability. The costs associated with these are very different, and will also vary by type of 
UMAS. Some questions to consider when estimating the UMAS, but specifically 
demonstrating the UMAS, are as follows: 


Level of Autonomy: 


a. At what level of autonomy is the UMAS designed to operate? 
b. How will the level of autonomy influence safety, reliability, and 
integration to other systems? 
Systems Integration: 


a. Will these demonstrations coincide with the design reviews or be 
separate events? 
b. What key system capabilities will your team want to demonstrate? 
Will you focus only on risky technology or demonstrate solutions to 
previously developed concepts? 
Full Operational Capability: 


a. Whois your audience? Depending on whether it is government or 
commercial this will play a huge factor in where and how you 
demonstrate. 

b. Will you need to create an operational scenario to show how the 
UMAS integrates into the current paradigm of its intended field? For 
example, will you need to have a mock battle, or create a queuing 
backlog at a distribution plant or border crossing? 


Conclusion 

In this section, we described unique considerations of Unmanned Autonomous 
Systems. In particular, life cycle models that help structure cost estimates, existing cost 
estimation methods, product work breakdown structures, and parametric models. These led 
to a case study that described an Army Unmanned Vehicle and a recommended approach 
for estimating the per unit life cycle cost. We concluded by discussing two unique 
considerations of estimating the cost of the UMAS—levels of autonomy, test and evaluation, 
and demonstration—that have the potential to significantly influence the complexities 
involved with transitioning a UMAS into operation. 
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As the UMAS continue to be developed and deployed into operation we anticipate 
the maturity and accuracy of estimating their costs will similarly increase. At the moment, 
reliance on complete work breakdown structures, comparisons with historical data, and 
utilization of existing parametric cost models can provide a reliable estimation process that 
can be used to develop realistic cost targets. 
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Research Questions 


From an acquisition perspective, what inadequacies exist, if any, with 
the toolsand methodsused to produce cost estimates for emergent 
UMAS technology? 


From a monetary and implementation view point, what are the hidden 
costs of UMAS - specifically when the system has left production and is 
placed into service? 
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Levels of Autonomy 





1 Human Operated 


2 Human Delegated 


3 Human Supervised 


4 Fully Autonomous 


A human operator makes all decisions. The system has no 
autonomous control of its environment although it may have 
information-only responses to sensed data. 


The Vehicle can perform many functions independently of human 
control when delegated to do so. This level encompasses automatic 
controls, engine controls, and other low-level automation that must 
activated ordeactivated by human input and must act in mutual 
exclusion of human operation. 


The system can perform a wide variety of activities when given top- 
level permissions of direction by a human. Both the human and the 
system can initiate behaviors based on sensed data, but the system 
can do so only if within the scope of its currently directed tasks. 


The system receives goals from humans and translates them into tasks 
to be performed without human interaction. A human could still 
enter the loop in an emergency orchange the goals, although in 
practice there may be significant time delays before human 
intervention occurs. 


Unmanned systems integrated roadmap FY2013-2038. (2013). Washington, D.C.: Department of Defense. 


Levels of Functionality 





Capability Comments 
[fo | Pre-programmed A software radio 


Fs Chooses Waveform According to Goal. Requires 
Goal Driven : 
Environment Awareness. 


[2 | Context Awareness Knowledge of What the User is Trying to Do 
Radio Aware Knowledge of Radio and Network Components, 
Environment Models 
Analyze Situation (Level 2& 3) to Determine Goals 
= Capable.of Planning (QoS, power), Follows Prescribed Plans 





Conducts Negotiations | Settle on a Plan with Another Radio 


Autonomously Determines Structure of 
Learns Environment x 
Environment 


Adapts Plans Generates New Goals 
Adapts Protocols Proposes and Negotiates New Protocols 


Mitola, J. “Cognitive Radio: An Integrated Agent Architecture for Software Defined Radio,” PhD Dissertation, Royal Institute of 
Technology, Sweden, May 2000. 
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Cost Estimation Methods 














pan lannin\ 9 & Preliminary Detail Design & Production 














Sees ee 
3 77 
a 


\ . 
NN Direct Engineering & 
AC Manufacturing Estimates 


Sart Program Program gram 











‘Systems & Industrial 
A Engineering 


> THE UNIVERSITY OF ARIZONA. 





Life Cycle Cost Hierarchy 


UAS Cost 
Development eessrree 
Hardware Facilities 
Software Personnel 
caeeeartg Maintenance 


Mission 


Production : 
Operations 
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Weight and Performance Metrics 








Payload Weight ( m) 
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Sik S10K = «S100K = SIM S10 St00M 1B Sik S10K—$100K SM SY0M_— 100M SHB 
Cost, $FY02 Cost, $FYO2 5 
Figure 4. UAV Capability Metric: Weight v. Cost! Figure 5. UAV Performance Metric: Endurance y. Cost 


UAV Roadmap, “Unmanned Aircraft Systems Roadmap: 2005-2030,” Office of the Secretary of Defense, August 4, 2005. 
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Proposed Methodology 


Cost (convert all individual outputs to $K) 





= (SEER-HDR) + (COCOMO II) + (COSYSMO) + (WeightBased CER) + (Per fomanceBased CER) 


Hardware - Best understood space with 
respects to UMAS. 


Software - Most complex space at both the 
lowest levels of UMAS WBS, and at integration of 
sub-systems 


Systems Engineering/ Program 
Management - Not asprecisely quantified 
asother effort categories. 


Weight Based CERs - Different UMAS will 
require different goals. Less may not always be 
betterin thiscategory. 


Performance Based CERs - This 
requirement will allow a system’sneed to 
remain the main driver, preventing “scope 
creep.” 


WBS Element 
UMAS System. 

Vehicle 

Vehicle integration 
Vehicle Sub-systems: 
‘Autonomous 
Capabilities 

Vehicle Electronics 
Navigation Capabilities 
Communications 
Remote Control System 
Ground Control Center 
Subsystem 


Operator Control Unit 
(OCU) Subsystem 


SEER- HOR 


cocom 


W 
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cosrso 


Weight 


Performance 
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» Software - Programming for operational environment 
interaction aswell asadapting and evolving may be 
the biggest challenge for Autonomy. 





¢ Test & Evaluation - We currently test UMS in similar 
fashion to MS. ForUMAS we will need to collect 
different data points, change interpretations, create 
autonomous test environments, and change 
paradigms. 





> HRI and MUM-T- Focusing on integration of UMAS 
with the end userand the operating environment. 
Issues are current human capacity, cultural 
acceptance, ethical dilemmas, most of the 
engineering “-ilities” 
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Potential Solutions Software Size - OOFP 








Object Oriented Function Points (OOFP) - Suggested Augmentation for COCOMO II 

COCOMO II curently uses the Intemational Function Point Users Group (IPFUG) standard for 
unadjusted function points (UFP). Traditional function point estimators view the final function 
point count through the lens of the end-user or the system itself, and what isnot accounted for 
in that viewpoint is the SOSor lens of the customer. OOFPscan be used to calculate size that 
encompasses both the individual system and its integration, interoperability, and capacity as 
an SOS- creating a more robust estimation. 











ohuctalutey ese) Uln(eya mi EAA=D) 








Test, Validation, Evaluation, and 
Demonstation (T-VED) - Suggested 
Augmentation for COSYSMO 

This cost driver rates the scale of requirements 
test worthiness at each level of the system. As the 
source of test worthiness increases the effort 
required to test, validate, evaluate, or 
demonstrate a requirement is lessened. 











No TVED TVED methods Current TVED 
methods are being methodsare 
currently developed and available and 
available to should be meet varying 
certify employable levels of 
requirement withinthe near — standards. 


success, Requires term (0-2 years). 
full development 
of any TVED. 





Pgh] Ven righ 
Curent TVED TVED methods 
methodsare in are proven and 
place and are reliable. These 


standard methodsare also 

compliant. consistent with 
respective 
standards. 


Potential Solution HRI-T 























Human Robot Interaction and Teaming (HRI-T) - Suggested 


Augmentation for COSYSMO 


This cost driver counts the number of input/interactions required between a 
system and the number of unique users/teams that ensure mission success. As 
the number of counts dec reases for HRI the effort estimation from a systems 
integration perspective increased. This's inversely applied for MUM-T, foras 
teaming capabilities increase (or the number of other systems it successfully 
cooperates with increase) the effort to integrate also increases. 
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‘System 
Poigiealy 
dependent on 
human/user 
interaction. 

1 

MUM-T System currently 
existsin a team 
with a singular 
manned system 
with established 
procedures. 


System requires 
interaction 
intermittently 
throughout total 
mission profile. 
2-3 

System curently 
exists in a team 
with a multiple 
manned systems 
with established 
procedures. 


System requires 
interaction only in 
critical phases of 
a mission profile. 


System iscapable After initial 

ofcompleting _callibration system 

mission without requires zero 

interaction. interaction during 
a mission. 


System exists in a 
team with 
manned and 
unmanned 
systems; all 
systems are 
controlled by 
humans. 










System exists in a 
team with 
manned and 


Team exists in a 
swam with 
mission 





unmanned parameters 
systems;some _calibrated prior to 
systems may execution. 


controlled by 
humans. 


Case Studies: Squad Maneuver 


Equipment Transport, Autonomous 
Mine Detection System 






The SMETwill lighten the Warfighter’s load and 
sustain the force during operations. The SMETwill 
maneuver with the dismounted force and enable 
Warfighters to conduct continuous operations 
without the individual Warfighter canying the 
equipment required to conduct 96 hours of 
dismounted operations. (Roberson, 2014) 





The AMDS is a team of 3 unique autonomous robots 
that when deployed together will provide the Amy 
with unprecedented capability to detect, mark, and 
neutralize explosive hazards in virtually all 
environments. The platform forthe collection robots 
isthe MTRS and includes cutting edge detection 
technology like AN/PSS-14 and Ground Penetrating 
Radar (GPR). (Army, 2015) 





Case Studies: Autonomous Mobility 


Appliqué System, Route Clearance 
Interrogation System 





RCIS doesnot procure any additional platforms, it will 
utilize existing HMEEs and RESETRG-31s. RCIS main 
purpose isto develop and field a Semi-Autonomous 
Control Capability that provides standoff interrogation 
and neutralization capabilities for Route Clearance. 
(Roberson, 2014) 





[RCIS} TYPE 1 [RCIs)TYPE2 


NPV PCY 





APY Type I PCY Type MRAP 


The AMAS interfaces between the “Autonomy Kit” 
and mission payload and the host platform's 
environment - enabling manned vehicles to be 


operated with autonomous capabilities. (Roberson, 
2014) 
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Cost Distributions for Army Systems 


AMAS-ACO 








HE LINIVERSITY OF ARIZONA. 


Systems & Industrial 


Case Study 


10% 30% 
5-37% 35-65% 
15% 45% 


RIDGE - The individual story line foreach 
UMAS factors into the data’s distribution. A 
new industry standard for systems with 
autonomy in the RDT&E phase could be 
expected to be more than 10% 


Procurement - Three of the UMASare re- 
using existing vehicular platforms keeping 
the allocation smallerthan SMET, which isa 
completely new system. As new systems 
are not COTS based we should see 
Procurement become more than 30% 





0% 0% 60% 
0% 0-6% 23-60% 
0% 0% 40% 


MILPER/MILC ON - most programs assume 
that their system is engineered to “fit-in” to the 
existing infrastructure and existing operating 
environment. This assumption will have to 
change asautonomy becomes prevalent. And 
we could see this cost category become utilized 
more often. 


O&M/O8&S - The O&M cost element isthe 
hardest element to estimate for. AsUMAS 

infra structure is built and exiting systems are 
supportive of this technology we will see a 
decrease in allocation - which does not equate 
to a smaller price tag. 
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hse ty BB cn Demet SAA dco 
Gap Analysis tool used when a military requirement is generated by 
one of the services 


Used unconventionally in this research asa way of analyzing how UMAS 
will impact these same areasafter implementation 


» Doctrine - Capacity exists to generate early concept style doctrine. Current 
method isto issue emergent systems and let Soldiers innovate through trial and 
error (sometimes in real-wond situations) 


» Organization, Training, Leadership , Personnel - Recommend initially exploring an 
Amny Special Forces structure of humansto robot ratio. Eventually expanding to 
regular Soldiers where incoming candidates train with their systems at entry level 
training then both Soldier are aligned by specialty 


‘Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities, and Policy 
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» Materiel - adjusting our understanding of how emergent systems will be mass produced 
should impact how we eventually field UMAS 


» Facilities- Recommend initial focus be on upgrading Test and Evaluation Centers (Yuma 
Proving Grounds, White Sands Missile Range, etc.) Secondly upgrade national military 
training centers (this could be both physical training infrastructure and with smulation 
training capacity) 


» Policy/ Acquisition - newest change is dated 7 J anuary 2015 - not able to quantify 
impact yet; however, this new policy introduced 4 unique acquisition strategies 
that could empower Project managers to be more agile in the process. 
(Hardware, Software, Incrementally deployed Software, and accelerated) 


» Policy/ Implementation - Focusison ethical use, rule of law, and unintended 
consequences. Provoke thought - the systems we build/use are never perfect. 
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Next Steps in Research 


Autonomous Model Validation 
Continue to seek program data to 
build and validate a parametric 
model 


Focuson critical areas of Test & 
Evaluation, User and System of 
Systems integration, and Sustainment 





Discuss and Contribute to a common Next Steps 


language for Autonomous Systems Delphi Survey based on research 


Expand portfolio 


aa ha 


Examine sub-system and sub- 
component autonomy 





Continue to refine the process for 
UMAS - embrace the coming 
paradigm shift 
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